Smaller, Faster, Greener: Compressing Pre-trained Code Models via Surrogate-Assisted Optimization

Large pre-trained models of code have been adopted to tackle many software engineering tasks and achieved excellent results. However, their large model size and expensive energy consumption prevent them from being widely deployed on developers' computers to provide real-time assistance. A recent study by Shi et al. can compress the pre-trained models into a small size. However, other important considerations in deploying models to have not been addressed: the model should have fast inference speed and minimal energy consumption. This requirement motivates us to propose Avatar, a novel approach that can reduce the model size as well as inference latency and energy consumption without compromising effectiveness (i.e., prediction accuracy). Avatar trains a surrogate model to predict the performance of a tiny model given only its hyperparameters setting. Moreover, Avatar designs a new fitness function embedding multiple key objectives, maximizing the predicted model accuracy and minimizing the model size, inference latency, and energy consumption. After finding the best model hyperparameters using a tailored genetic algorithm (GA), Avatar employs the knowledge distillation technique to train the tiny model. We evaluate Avatar and the baseline approach from Shi et al. on three datasets for two popular software engineering tasks: vulnerability prediction and clone detection. We use Avatar to compress models to a small size (3 MB), which is 160$\times$ smaller than the original pre-trained models. Compared with the original models, the inference latency of compressed models is significantly reduced on all three datasets. On average, our approach is capable of reducing the inference latency by 62$\times$, 53$\times$, and 186$\times$. In terms of energy consumption, compressed models only have 0.8 GFLOPs, which is 173$\times$ smaller than the original pre-trained models.Comment: 12 pages, a working-in-progress versio

Stealthy Backdoor Attack for Code Models

Code models, such as CodeBERT and CodeT5, offer general-purpose representations of code and play a vital role in supporting downstream automated software engineering tasks. Most recently, code models were revealed to be vulnerable to backdoor attacks. A code model that is backdoor-attacked can behave normally on clean examples but will produce pre-defined malicious outputs on examples injected with triggers that activate the backdoors. Existing backdoor attacks on code models use unstealthy and easy-to-detect triggers. This paper aims to investigate the vulnerability of code models with stealthy backdoor attacks. To this end, we propose AFRAIDOOR (Adversarial Feature as Adaptive Backdoor). AFRAIDOOR achieves stealthiness by leveraging adversarial perturbations to inject adaptive triggers into different inputs. We evaluate AFRAIDOOR on three widely adopted code models (CodeBERT, PLBART and CodeT5) and two downstream tasks (code summarization and method name prediction). We find that around 85% of adaptive triggers in AFRAIDOOR bypass the detection in the defense process. By contrast, only less than 12% of the triggers from previous work bypass the defense. When the defense method is not applied, both AFRAIDOOR and baselines have almost perfect attack success rates. However, once a defense is applied, the success rates of baselines decrease dramatically to 10.47% and 12.06%, while the success rate of AFRAIDOOR are 77.05% and 92.98% on the two tasks. Our finding exposes security weaknesses in code models under stealthy backdoor attacks and shows that the state-of-the-art defense method cannot provide sufficient protection. We call for more research efforts in understanding security threats to code models and developing more effective countermeasures.Comment: 18 pages, Under review of IEEE Transactions on Software Engineerin

Drug Target Prediction Based on the Herbs Components: The Study on the Multitargets Pharmacological Mechanism of Qishenkeli Acting on the Coronary Heart Disease

In this paper, we present a case study of Qishenkeli (QSKL) to research TCM's underlying molecular mechanism, based on drug target prediction and analyses of TCM chemical components and following experimental validation. First, after determining the compositive compounds of QSKL, we use drugCIPHER-CS to predict their potential drug targets. These potential targets are significantly enriched with known cardiovascular disease-related drug targets. Then we find these potential drug targets are significantly enriched in the biological processes of neuroactive ligand-receptor interaction, aminoacyl-tRNA biosynthesis, calcium signaling pathway, glycine, serine and threonine metabolism, and renin-angiotensin system (RAAS), and so on. Then, animal model of coronary heart disease (CHD) induced by left anterior descending coronary artery ligation is applied to validate predicted pathway. RAAS pathway is selected as an example, and the results show that QSKL has effect on both rennin and angiotensin II receptor (AT1R), which eventually down regulates the angiotensin II (AngII). Bioinformatics combing with experiment verification can provide a credible and objective method to understand the complicated multitargets mechanism for Chinese herbal formula

Mind Your Data! Hiding Backdoors in Offline Reinforcement Learning Datasets

A growing body of research works has focused on the Offline Reinforcement Learning (RL) paradigm. Data providers share large pre-collected datasets on which others can train high-quality agents without interacting with the environments. Such an offline RL paradigm has demonstrated effectiveness in many critical tasks, including robot control, autonomous driving, etc. A well-trained agent can be regarded as a software system. However, less attention is paid to investigating the security threats to the offline RL system. In this paper, we focus on a critical security threat: backdoor attacks. Given normal observations, an agent implanted with backdoors takes actions leading to high rewards. However, the same agent takes actions that lead to low rewards if the observations are injected with triggers that can activate the backdoor. In this paper, we propose Baffle (Backdoor Attack for Offline Reinforcement Learning) and evaluate how different Offline RL algorithms react to this attack. Our experiments conducted on four tasks and four offline RL algorithms expose a disquieting fact: none of the existing offline RL algorithms is immune to such a backdoor attack. More specifically, Baffle modifies $10\%$ of the datasets for four tasks (3 robotic controls and 1 autonomous driving). Agents trained on the poisoned datasets perform well in normal settings. However, when triggers are presented, the agents' performance decreases drastically by $63.6\%$, $57.8\%$, $60.8\%$ and $44.7\%$ in the four tasks on average. The backdoor still persists after fine-tuning poisoned agents on clean datasets. We further show that the inserted backdoor is also hard to be detected by a popular defensive method. This paper calls attention to developing more effective protection for the open-source offline RL dataset.Comment: 13 pages, 6 figure

Answer Summarization for Technical Queries: Benchmark and New Approach

Prior studies have demonstrated that approaches to generate an answer summary for a given technical query in Software Question and Answer (SQA) sites are desired. We find that existing approaches are assessed solely through user studies. There is a need for a benchmark with ground truth summaries to complement assessment through user studies. Unfortunately, such a benchmark is non-existent for answer summarization for technical queries from SQA sites. To fill the gap, we manually construct a high-quality benchmark to enable automatic evaluation of answer summarization for technical queries for SQA sites. Using the benchmark, we comprehensively evaluate the performance of existing approaches and find that there is still a big room for improvement. Motivated by the results, we propose a new approach TechSumBot with three key modules:1) Usefulness Ranking module, 2) Centrality Estimation module, and 3) Redundancy Removal module. We evaluate TechSumBot in both automatic (i.e., using our benchmark) and manual (i.e., via a user study) manners. The results from both evaluations consistently demonstrate that TechSumBot outperforms the best performing baseline approaches from both SE and NLP domains by a large margin, i.e., 10.83%-14.90%, 32.75%-36.59%, and 12.61%-17.54%, in terms of ROUGE-1, ROUGE-2, and ROUGE-L on automatic evaluation, and 5.79%-9.23% and 17.03%-17.68%, in terms of average usefulness and diversity score on human evaluation. This highlights that the automatic evaluation of our benchmark can uncover findings similar to the ones found through user studies. More importantly, automatic evaluation has a much lower cost, especially when it is used to assess a new approach. Additionally, we also conducted an ablation study, which demonstrates that each module in TechSumBot contributes to boosting the overall performance of TechSumBot.Comment: Accepted by ASE 202

Being a morning man has causal effects on the cerebral cortex: a Mendelian randomization study

IntroductionNumerous studies have suggested a connection between circadian rhythm and neurological disorders with cognitive and consciousness impairments in humans, yet little evidence stands for a causal relationship between circadian rhythm and the brain cortex.MethodsThe top 10,000 morningness-related single-nucleotide polymorphisms of the Genome-wide association study (GWAS) summary statistics were used to filter the instrumental variables. GWAS summary statistics from the ENIGMA Consortium were used to assess the causal relationship between morningness and variates like cortical thickness (TH) or surficial area (SA) on the brain cortex. The inverse-variance weighted (IVW) and weighted median (WM) were used as the major estimates whereas MR-Egger, MR Pleiotropy RESidual Sum and Outlier, leave-one-out analysis, and funnel-plot were used for heterogeneity and pleiotropy detecting.ResultsRegionally, morningness decreased SA of the rostral middle frontal gyrus with genomic control (IVW: β = −24.916 mm, 95% CI: −47.342 mm to −2.490 mm, p = 0.029. WM: β = −33.208 mm, 95% CI: −61.933 mm to −4.483 mm, p = 0.023. MR Egger: β < 0) and without genomic control (IVW: β = −24.581 mm, 95% CI: −47.552 mm to −1.609 mm, p = 0.036. WM: β = −32.310 mm, 95% CI: −60.717 mm to −3.902 mm, p = 0.026. MR Egger: β < 0) on a nominal significance, with no heterogeneity or no outliers.Conclusions and implicationsCircadian rhythm causally affects the rostral middle frontal gyrus; this sheds new light on the potential use of MRI in disease diagnosis, revealing the significance of circadian rhythm on the progression of disease, and might also suggest a fresh therapeutic approach for disorders related to the rostral middle frontal gyrus-related

A New Acoustic Emission Sensor Based Gear Fault Detection Approach

In order to reduce wind energy costs, prognostics and health management (PHM) of wind turbine is needed to reduce operations and maintenance cost of wind turbines. The major cost on wind turbine repairs is due to gearbox failure. Therefore, developing effective gearbox fault detection tools is important in the PHM of wind turbine. PHM system allows less costly maintenance because it can inform operators of needed repairs before a fault causes collateral damage happens to the gearbox. In this paper, a new acoustic emission (AE) sensor based gear fault detection approach is presented. This approach combines a heterodyne based frequency reduction technique with time synchronous average (TSA) and spectral kurtosis (SK) to process AE sensor signals and extract features as condition indictors for gear fault detection. Heterodyne techniques commonly used in communication are used to preprocess the AE signals before sampling. By heterodyning, the AE signal frequency is down shifted from MHz to below 50 kHz. This reduced AE signal sampling rate is comparable to that of vibration signals. The presented approach is validated using seeded gear tooth crack fault tests on a notational split torque gearbox. The approach presented in this paper is physics based and the validation results have showed that it could effectively detect the gear faults. 1

Natural attack for pre-trained models of code

Pre-trained models of code have achieved success in many important software engineering tasks. However, these powerful models are vulnerable to adversarial attacks that slightly perturb model inputs to make a victim model produce wrong outputs. Current works mainly attack models of code with examples that preserve operational program semantics but ignore a fundamental requirement for adversarial example generation: perturbations should be natural to human judges, which we refer to as naturalness requirement. In this paper, we propose ALERT (nAturaLnEss AwaRe ATtack), a black-box attack that adversarially transforms inputs to make victim models produce wrong outputs. Different from prior works, this paper considers the natural semantic of generated examples at the same time as preserving the operational semantic of original inputs. Our user study demonstrates that human developers consistently consider that adversarial examples generated by ALERT are more natural than those generated by the state-of-the-art work by Zhang et al. that ignores the naturalness requirement. On attacking CodeBERT, our approach can achieve attack success rates of 53.62%, 27.79%, and 35.78% across three downstream tasks: vulnerability prediction, clone detection and code authorship attribution. On GraphCodeBERT, our approach can achieve average success rates of 76.95%, 7.96% and 61.47% on the three tasks. The above outperforms the baseline by 14.07% and 18.56% on the two pre-trained models on average. Finally, we investigated the value of the generated adversarial examples to harden victim models through an adversarial fine-tuning procedure and demonstrated the accuracy of CodeBERT and GraphCodeBERT against ALERT-generated adversarial examples increased by 87.59% and 92.32%, respectively.Comment: To appear in the Technical Track of ICSE 202